Dynamic Data Placement Strategy in MapReduce-styled Data Processing Platform
نویسندگان
چکیده
منابع مشابه
Adaptive Dynamic Data Placement Algorithm for Hadoop in Heterogeneous Environments
Hadoop MapReduce framework is an important distributed processing model for large-scale data intensive applications. The current Hadoop and the existing Hadoop distributed file system’s rack-aware data placement strategy in MapReduce in the homogeneous Hadoop cluster assume that each node in a cluster has the same computing capacity and a same workload is assigned to each node. Default Hadoop d...
متن کاملParallel Data Processing in Dynamic Hybrid Computing Environment Using MapReduce
A novel MapReduce computation model in hybrid computing environment called HybridMR is proposed in the paper. Using this model, high performance cluster nodes and heterogeneous desktop PCs in Internet or Intranet can be integrated to form a hybrid computing environment. In this way, the computation and storage capability of largescale desktop PCs can be fully utilized to process large-scale dat...
متن کاملP2P-MapReduce: Parallel data processing in dynamic Cloud environments
MapReduce is a programming model for parallel data processing widely used in Cloud computing environments. Current MapReduce implementations are based on centralized master-slave architectures that do not cope well with dynamic Cloud infrastructures, like a Cloud of clouds, in which nodes may join and leave the network at high rates. We have designed an adaptive MapReduce framework, called P2P-...
متن کاملA Simplified Data Processing in MapReduce
For processing and generating large data sets we use MapReduce as a programming model and their associated implementations. A map function is specified by a user to generate a set of intermediate key/value pairs from processes a key/value pair. The warehousing systems existing based MapReduce are not specially optimized for time-based big data analysis applications. Such applications have two c...
متن کاملEfficient Big Data Processing in Hadoop MapReduce
This tutorial is motivated by the clear need of many organizations, companies, and researchers to deal with big data volumes efficiently. Examples include web analytics applications, scientific applications, and social networks. A popular data processing engine for big data is Hadoop MapReduce. Early versions of Hadoop MapReduce suffered from severe performance problems. Today, this is becoming...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: DEStech Transactions on Engineering and Technology Research
سال: 2016
ISSN: 2475-885X
DOI: 10.12783/dtetr/ssme-ist2016/3929